Air Quality in Southampton (UK)
Exploring the data
Ben Anderson (b.anderson@soton.ac.uk @dataknut)
Last run at: 2020-03-12 12:18:39 (Europe/London)
1 About
1.1 Contributions
Please note that authorship is alphabetical. Contributions are listed below - see github for details and who to blame for what :-).
- Ben Anderson (b.anderson@soton.ac.uk
@dataknut)
1.3 Citation
If you wish to refer to any of the material from this report please cite as:
- Anderson, B., (2020) Air Quality in Southampton (UK): Exploring the data , University of Southampton: Southampton, UK.
Report circulation:
- Public
Report purpose:
- to explore (official) Southampton City Council Air Quality data (http://southampton.my-air.uk)
This work is (c) 2020 the University of Southampton.
2 Introduction
Data downloaded from http://southampton.my-air.uk. See also https://www.southampton.gov.uk/environmental-issues/pollution/air-quality/.
Southampton City Council collects various forms of air quality data at the sites shown in 2.1. WHO publishes information on the health consequences and “acceptable” exposure levels for each of these.
lDT <- data.table::melt(origDataDT, id.vars = c("site", "dateTimeUTC"), measure.vars = c("co",
"no2", "nox", "oz", "pm10", "pm2_5", "so2"), value.name = "value" # varies
)
lDT[, `:=`(obsDate, lubridate::date(dateTimeUTC))]
# remove NA lDT <- lDT[!is.na(value)]t <- lDT[!is.na(value), .(from = min(dateTimeUTC), to = max(dateTimeUTC), nObs = .N), keyby = .(site,
variable)]
kableExtra::kable(t, caption = "Dates data != NA available by site and measure", digits = 2) %>%
kable_styling()| site | variable | from | to | nObs |
|---|---|---|---|---|
| Southampton - A33 Roadside (near docks, AURN site) | no2 | 2016-01-26 11:00:00 | 2020-03-11 13:00:00 | 34980 |
| Southampton - A33 Roadside (near docks, AURN site) | pm10 | 2016-01-04 13:00:00 | 2020-03-11 12:00:00 | 33485 |
| Southampton - Background (near city centre, AURN site) | no2 | 2016-01-01 00:00:00 | 2020-03-11 13:00:00 | 29812 |
| Southampton - Background (near city centre, AURN site) | nox | 2016-01-01 00:00:00 | 2019-01-01 00:00:00 | 20026 |
| Southampton - Background (near city centre, AURN site) | oz | 2016-01-01 00:00:00 | 2020-03-11 13:00:00 | 29490 |
| Southampton - Background (near city centre, AURN site) | pm10 | 2016-01-01 00:00:00 | 2020-03-11 13:00:00 | 26065 |
| Southampton - Background (near city centre, AURN site) | pm2_5 | 2016-01-01 00:00:00 | 2020-03-11 13:00:00 | 28273 |
| Southampton - Background (near city centre, AURN site) | so2 | 2016-01-01 00:00:00 | 2020-03-11 13:00:00 | 29306 |
| Southampton - Onslow Road (near RSH) | no2 | 2016-01-01 00:00:00 | 2020-03-11 13:00:00 | 35439 |
| Southampton - Onslow Road (near RSH) | nox | 2016-01-01 00:00:00 | 2020-03-11 13:00:00 | 35443 |
| Southampton - Victoria Road (Woolston) | no2 | 2016-01-01 00:00:00 | 2020-03-11 08:00:00 | 27116 |
| Southampton - Victoria Road (Woolston) | nox | 2016-01-01 00:00:00 | 2020-03-11 08:00:00 | 27116 |
3 Summarise data
Summarise previously downloaded and processed data… Note that this may not be completely up to date.
t <- lDT[, .(mean = mean(value, na.rm = TRUE)), keyby = .(site, variable)]
kableExtra::kable(t, caption = "Mean values per site (NaN indicates not measured)") %>% kable_styling()| site | variable | mean |
|---|---|---|
| Southampton - A33 Roadside (near docks, AURN site) | co | NaN |
| Southampton - A33 Roadside (near docks, AURN site) | no2 | 35.571547 |
| Southampton - A33 Roadside (near docks, AURN site) | nox | NaN |
| Southampton - A33 Roadside (near docks, AURN site) | oz | NaN |
| Southampton - A33 Roadside (near docks, AURN site) | pm10 | 18.928350 |
| Southampton - A33 Roadside (near docks, AURN site) | pm2_5 | NaN |
| Southampton - A33 Roadside (near docks, AURN site) | so2 | NaN |
| Southampton - Background (near city centre, AURN site) | co | NaN |
| Southampton - Background (near city centre, AURN site) | no2 | 29.445508 |
| Southampton - Background (near city centre, AURN site) | nox | 50.789064 |
| Southampton - Background (near city centre, AURN site) | oz | 40.877009 |
| Southampton - Background (near city centre, AURN site) | pm10 | 18.079098 |
| Southampton - Background (near city centre, AURN site) | pm2_5 | 12.009391 |
| Southampton - Background (near city centre, AURN site) | so2 | 2.970521 |
| Southampton - Bitterne | co | NaN |
| Southampton - Bitterne | no2 | NaN |
| Southampton - Bitterne | nox | NaN |
| Southampton - Bitterne | oz | NaN |
| Southampton - Bitterne | pm10 | NaN |
| Southampton - Bitterne | pm2_5 | NaN |
| Southampton - Bitterne | so2 | NaN |
| Southampton - Onslow Road (near RSH) | co | NaN |
| Southampton - Onslow Road (near RSH) | no2 | 42.379229 |
| Southampton - Onslow Road (near RSH) | nox | 87.529707 |
| Southampton - Onslow Road (near RSH) | oz | NaN |
| Southampton - Onslow Road (near RSH) | pm10 | NaN |
| Southampton - Onslow Road (near RSH) | pm2_5 | NaN |
| Southampton - Onslow Road (near RSH) | so2 | NaN |
| Southampton - Redbridge | co | NaN |
| Southampton - Redbridge | no2 | NaN |
| Southampton - Redbridge | nox | NaN |
| Southampton - Redbridge | oz | NaN |
| Southampton - Redbridge | pm10 | NaN |
| Southampton - Redbridge | pm2_5 | NaN |
| Southampton - Redbridge | so2 | NaN |
| Southampton - Victoria Road (Woolston) | co | NaN |
| Southampton - Victoria Road (Woolston) | no2 | 38.925457 |
| Southampton - Victoria Road (Woolston) | nox | 80.190670 |
| Southampton - Victoria Road (Woolston) | oz | NaN |
| Southampton - Victoria Road (Woolston) | pm10 | NaN |
| Southampton - Victoria Road (Woolston) | pm2_5 | NaN |
| Southampton - Victoria Road (Woolston) | so2 | NaN |
Table 3.1 gives an indication of the availability of the different measures.
4 Analysis
In this section we present graphical analysis of the previoulsy downloaded data. Note this is just a snapshot of the data available.
4.1 Nitrogen Dioxide
yLab <- "Nitrogen Dioxide (ug/m3)"
dt <- lDT[variable == "no2"]
t <- dt[, .(mean = mean(value, na.rm = TRUE), sd = sd(value, na.rm = TRUE), min = min(value,
na.rm = TRUE), max = max(value, na.rm = TRUE)), keyby = .(site)]
kableExtra::kable(t, caption = "Summary of NO2 data") %>% kable_styling()| site | mean | sd | min | max |
|---|---|---|---|---|
| Southampton - A33 Roadside (near docks, AURN site) | 35.57155 | 24.80918 | 0.0 | 164.3 |
| Southampton - Background (near city centre, AURN site) | 29.44551 | 16.95406 | 2.3 | 144.2 |
| Southampton - Bitterne | NaN | NA | Inf | -Inf |
| Southampton - Onslow Road (near RSH) | 42.37923 | 21.17712 | 1.2 | 241.3 |
| Southampton - Redbridge | NaN | NA | Inf | -Inf |
| Southampton - Victoria Road (Woolston) | 38.92546 | 23.85054 | -1.3 | 287.5 |
Table 4.1 suggests that there may be a few (5) negative values. These are summarised in 4.2 while Figure 4.1 shows the availability and levels of the pollutant data over time.
t <- head(dt[value < 0], 10)
kableExtra::kable(t, caption = "Negative NO2 values (up to first 10)") %>% kable_styling()| site | dateTimeUTC | variable | value | obsDate |
|---|---|---|---|---|
| Southampton - Victoria Road (Woolston) | 2016-01-29 04:00:00 | no2 | -0.1 | 2016-01-29 |
| Southampton - Victoria Road (Woolston) | 2016-11-27 05:00:00 | no2 | -0.5 | 2016-11-27 |
| Southampton - Victoria Road (Woolston) | 2016-11-27 06:00:00 | no2 | -1.1 | 2016-11-27 |
| Southampton - Victoria Road (Woolston) | 2017-02-06 00:00:00 | no2 | -0.2 | 2017-02-06 |
| Southampton - Victoria Road (Woolston) | 2017-02-06 01:00:00 | no2 | -1.3 | 2017-02-06 |
t <- table(dt[value < 0]$site)
kableExtra::kable(t, caption = "Negative NO2 values (count by site)") %>% kable_styling()| Var1 | Freq |
|---|---|
| Southampton - Victoria Road (Woolston) | 5 |
# dt,xvar, yvar,fillVar, yLab
p <- makeTilePlot(dt, xVar = "dateTimeUTC", yVar = "site", fillVar = "value", yLab = yLab)
p
Figure 4.1: Nitrogen Dioxide data availability and levels over time
Figure 4.2 shows daily mean values for all sites over time and includes smoother trend lines for each site.
plotDT <- dt[!is.na(value), .(mean = mean(value, na.rm = TRUE)),
keyby = .(obsDate, site)]
p <- makeDotPlot(plotDT,
xVar = "obsDate",
yVar = "mean",
byVar = "site",
yLab = paste0("Mean daily ", yLab)
)
p <- p +
geom_smooth() + # add smoothed line
labs(caption = "Trend line = Generalized additive model (gam) with integrated smoothness estimation")
if(doPlotly){
p
plotly::ggplotly(p + xlim(xlimMinDate, xlimMaxDate)) # interactive, xlimited # interactive
} else {
p
}
Figure 4.2: Nitrogen Dioxide levels, Southampton (daily mean
dt <- dt[dateTimeUTC > oneYearAgo]
t <- dt[value > hourlyNo2Threshold_WHO][order(-value)]
kableExtra::kable(caption = paste0("Values greater than WHO threshold (NO2 > ", hourlyNo2Threshold_WHO,
", last 12 months)"), head(t, 10)) %>% kable_styling()| site | dateTimeUTC | variable | value | obsDate |
|---|---|---|---|---|
p <- makeDotPlot(dt[!is.na(value)], xVar = "dateTimeUTC", yVar = "value", byVar = "site", yLab = yLab)
p <- p + geom_hline(yintercept = hourlyNo2Threshold_WHO) + labs(caption = "Reference line = WHO hourly threshold")
plotly::ggplotly(p) # for interactionFigure 4.3: Nitrogen Dioxide levels, Southampton (hourly, last 12 months)
Figure 4.3 shows hourly values for all sites for the last 12 months. In this period there were 0 hours when the hourly Nitrogen Dioxide level breached the relevant WHO hourly threshold (200). The worst 10 cases (if any) are shown in Table 4.3.
Clearly there are winter peaks, the mean daily values show less variance (and less extremes) than the hourly data and there has also been a decreasing trend over time.
4.2 PM 10
PM 10 data: has more sensors and wider coverage than PM2.5
yLab <- "PM 10 (ug/m3)"
dt <- lDT[variable == "pm10"]
t <- dt[, .(mean = mean(value, na.rm = TRUE), sd = sd(value, na.rm = TRUE), min = min(value,
na.rm = TRUE), max = max(value, na.rm = TRUE)), keyby = .(site)]
kableExtra::kable(t, caption = "Summary of pm10 data") %>% kable_styling()| site | mean | sd | min | max |
|---|---|---|---|---|
| Southampton - A33 Roadside (near docks, AURN site) | 18.92835 | 14.25055 | 0.0 | 761.6 |
| Southampton - Background (near city centre, AURN site) | 18.07910 | 11.32865 | -3.5 | 344.1 |
| Southampton - Bitterne | NaN | NA | Inf | -Inf |
| Southampton - Onslow Road (near RSH) | NaN | NA | Inf | -Inf |
| Southampton - Redbridge | NaN | NA | Inf | -Inf |
| Southampton - Victoria Road (Woolston) | NaN | NA | Inf | -Inf |
Table 4.4 suggests that there may be a few (19) negative values. These are shown in 4.5 while 4.4 shows data availability and PM 10 levels over time at each site.
t <- head(dt[value < 0], nrow(dt[value < 0]))
kableExtra::kable(head(t), caption = "Negative PM10 values - first 6") %>% kable_styling()| site | dateTimeUTC | variable | value | obsDate |
|---|---|---|---|---|
| Southampton - Background (near city centre, AURN site) | 2016-04-16 10:00:00 | pm10 | -0.9 | 2016-04-16 |
| Southampton - Background (near city centre, AURN site) | 2016-06-12 06:00:00 | pm10 | -2.0 | 2016-06-12 |
| Southampton - Background (near city centre, AURN site) | 2016-06-12 07:00:00 | pm10 | -2.1 | 2016-06-12 |
| Southampton - Background (near city centre, AURN site) | 2017-06-11 03:00:00 | pm10 | -1.2 | 2017-06-11 |
| Southampton - Background (near city centre, AURN site) | 2017-06-30 11:00:00 | pm10 | -0.1 | 2017-06-30 |
| Southampton - Background (near city centre, AURN site) | 2017-07-04 02:00:00 | pm10 | -0.6 | 2017-07-04 |
p <- makeTilePlot(lDT[variable == "pm10"], xVar = "dateTimeUTC", yVar = "site", fillVar = "value",
yLab = yLab)
p
Figure 4.4: Availability and level of PM 10 data over time
plotDT <- dt[!is.na(value), .(mean = mean(value, na.rm = TRUE)),
keyby = .(obsDate, site)]
extremePm10Daily <- plotDT[mean > dailyPm10Threshold_WHO][order(-mean)]
kableExtra::kable(caption = paste0("10 highest values greater than WHO threshold (PM 10 > ",
dailyPm10Threshold_WHO , ")"),
digits = 2,
head(extremePm10Daily, 10)) %>%
kable_styling()| obsDate | site | mean |
|---|---|---|
| 2016-08-09 | Southampton - A33 Roadside (near docks, AURN site) | 144.48 |
| 2017-06-19 | Southampton - A33 Roadside (near docks, AURN site) | 143.66 |
| 2017-05-05 | Southampton - A33 Roadside (near docks, AURN site) | 88.94 |
| 2016-09-19 | Southampton - A33 Roadside (near docks, AURN site) | 88.46 |
| 2016-11-30 | Southampton - A33 Roadside (near docks, AURN site) | 79.25 |
| 2016-03-12 | Southampton - A33 Roadside (near docks, AURN site) | 72.00 |
| 2017-01-22 | Southampton - A33 Roadside (near docks, AURN site) | 66.86 |
| 2019-02-27 | Southampton - Background (near city centre, AURN site) | 64.67 |
| 2019-02-27 | Southampton - A33 Roadside (near docks, AURN site) | 60.60 |
| 2019-04-17 | Southampton - Background (near city centre, AURN site) | 58.37 |
p <- makeDotPlot(plotDT,
xVar = "obsDate",
yVar = "mean",
byVar = "site",
yLab = paste0("Mean daily ", yLab)
)
p <- p +
geom_hline(yintercept = dailyPm10Threshold_WHO) +
geom_smooth() + # add smoothed line
labs(caption = "Trend line = Generalized additive model (gam) with integrated smoothness estimation\nReference line = WHO PM10 mean daily threshold")
if(doPlotly){
p
plotly::ggplotly(p + xlim(xlimMinDate, xlimMaxDate)) # interactive, xlimited # interactive
} else {
p
}
Figure 4.5: PM10 levels, Southampton (daily mean)
nDaysOverThreshold <- uniqueN(extremePm10Daily$obsDate)
nDays <- uniqueN(plotDT$obsDate) # need to count days not site-daysFigure 4.5 shows daily values for all sites across the entire dataset and indicates the 19 days (1.3%) that breached the WHO PM10 daily mean exposure threshold (50) - see Table 4.6.
dt <- dt[dateTimeUTC > oneYearAgo]
t <- dt[value > 100][order(-value)]
kableExtra::kable(caption = "10 highest hourly values (PM 10 > 100)", head(t)) %>% kable_styling()| site | dateTimeUTC | variable | value | obsDate |
|---|---|---|---|---|
| Southampton - Background (near city centre, AURN site) | 2019-10-27 20:00:00 | pm10 | 252.5 | 2019-10-27 |
| Southampton - Background (near city centre, AURN site) | 2019-12-02 19:00:00 | pm10 | 174.6 | 2019-12-02 |
| Southampton - Background (near city centre, AURN site) | 2019-10-27 19:00:00 | pm10 | 162.2 | 2019-10-27 |
| Southampton - Background (near city centre, AURN site) | 2019-12-02 18:00:00 | pm10 | 114.3 | 2019-12-02 |
| Southampton - Background (near city centre, AURN site) | 2019-12-02 21:00:00 | pm10 | 109.1 | 2019-12-02 |
| Southampton - Background (near city centre, AURN site) | 2019-03-29 22:00:00 | pm10 | 102.8 | 2019-03-29 |
p <- makeDotPlot(dt[!is.na(value)], xVar = "dateTimeUTC", yVar = "value", byVar = "site", yLab = yLab)
p <- p + labs(caption = "NB: There is no WHO PM10 hourly threshold")
if (doPlotly) {
p
plotly::ggplotly(p + xlim(xlimMinDateTime, xlimMaxDateTime)) # interactive, xlimited # interactive
} else {
p
}
Figure 4.6: PM10 levels, Southampton (hourly, last 12 months)
Figure 4.6 shows hourly PM 10 values for all sites over the last 12 months and suggests there may be outliers (see Table 4.7).
4.3 PM 2.5
yLab <- "PM 2.5 (ug/m3)"
dt <- lDT[variable == "pm2_5"]
t <- dt[!is.na(value), .(mean = mean(value, na.rm = TRUE), sd = sd(value, na.rm = TRUE), min = min(value,
na.rm = TRUE), max = max(value, na.rm = TRUE)), keyby = .(site)]
kableExtra::kable(t, caption = "Summary of pm2_5 data") %>% kable_styling()| site | mean | sd | min | max |
|---|---|---|---|---|
| Southampton - Background (near city centre, AURN site) | 12.00939 | 9.47876 | -4 | 289.2 |
Table 4.8 suggests that there may be a few (89) negative values. These are shown in Table 4.9 while Figure 4.7 shows data availability and PM 2.5 levels over time at each site.
t <- head(dt[value < 0], nrow(dt[value < 0]))
kableExtra::kable(head(t), caption = "Negative pm2_5 values - first 6") %>% kable_styling()| site | dateTimeUTC | variable | value | obsDate |
|---|---|---|---|---|
| Southampton - Background (near city centre, AURN site) | 2016-01-02 17:00:00 | pm2_5 | -0.8 | 2016-01-02 |
| Southampton - Background (near city centre, AURN site) | 2016-01-04 03:00:00 | pm2_5 | -0.9 | 2016-01-04 |
| Southampton - Background (near city centre, AURN site) | 2016-01-04 23:00:00 | pm2_5 | -0.3 | 2016-01-04 |
| Southampton - Background (near city centre, AURN site) | 2016-01-05 01:00:00 | pm2_5 | -1.0 | 2016-01-05 |
| Southampton - Background (near city centre, AURN site) | 2016-01-09 22:00:00 | pm2_5 | -0.3 | 2016-01-09 |
| Southampton - Background (near city centre, AURN site) | 2016-01-09 23:00:00 | pm2_5 | -0.9 | 2016-01-09 |
p <- makeTilePlot(dt, xVar = "dateTimeUTC", yVar = "site", fillVar = "value", yLab = yLab)
p
Figure 4.7: Availability and level of PM 10 data over time
plotDT <- dt[!is.na(value), .(mean = mean(value, na.rm = TRUE)),
keyby = .(obsDate, site)]
extremePm25Daily <- plotDT[mean > dailyPm2.5Threshold_WHO][order(-mean)]
kableExtra::kable(caption = paste0("6 highest values greater than WHO threshold (PM 2.5 > ",
dailyPm2.5Threshold_WHO , ")"),
digits = 2,
head(extremePm25Daily)) %>%
kable_styling()| obsDate | site | mean |
|---|---|---|
| 2016-03-12 | Southampton - Background (near city centre, AURN site) | 64.79 |
| 2019-04-17 | Southampton - Background (near city centre, AURN site) | 49.04 |
| 2016-03-13 | Southampton - Background (near city centre, AURN site) | 47.79 |
| 2019-04-07 | Southampton - Background (near city centre, AURN site) | 46.90 |
| 2019-02-27 | Southampton - Background (near city centre, AURN site) | 43.64 |
| 2019-03-29 | Southampton - Background (near city centre, AURN site) | 43.60 |
p <- makeDotPlot(plotDT,
xVar = "obsDate",
yVar = "mean",
byVar = "site",
yLab = paste0("Mean daily ", yLab)
)
p <- p +
geom_hline(yintercept = dailyPm2.5Threshold_WHO) +
geom_smooth() + #add smoothed line
labs(caption = "Trend line = Generalized additive model (gam) with integrated smoothness estimation\nReference line = WHO daily PM2.5 threshold")
if(doPlotly){
p
plotly::ggplotly(p + xlim(xlimMinDate, xlimMaxDate)) # interactive, xlimited # interactive
} else {
p
}
Figure 4.8: PM2_5 levels, Southampton (daily mean)
nDaysOverThreshold <- uniqueN(extremePm25Daily$obsDate)
nDays <- uniqueN(plotDT$obsDate) # need to count days not site-daysFigure 4.8 shows daily values for all sites across the dataset and indicates that the WHO PM2_5 daily mean exposure threshold (25) was breached on 83 days (6.9 %). The 6 worst cases are shown in Table 4.10.
dt <- dt[dateTimeUTC > oneYearAgo]
t <- dt[value > 50][order(-value)]
kableExtra::kable(caption = "Extreme hourly values (PM 2.5 > 50, last 12 months, worst 6)", head(t)) %>%
kable_styling()| site | dateTimeUTC | variable | value | obsDate |
|---|---|---|---|---|
| Southampton - Background (near city centre, AURN site) | 2019-10-27 20:00:00 | pm2_5 | 239.1 | 2019-10-27 |
| Southampton - Background (near city centre, AURN site) | 2019-12-02 19:00:00 | pm2_5 | 167.6 | 2019-12-02 |
| Southampton - Background (near city centre, AURN site) | 2019-10-27 19:00:00 | pm2_5 | 140.4 | 2019-10-27 |
| Southampton - Background (near city centre, AURN site) | 2019-12-02 18:00:00 | pm2_5 | 109.9 | 2019-12-02 |
| Southampton - Background (near city centre, AURN site) | 2019-12-02 21:00:00 | pm2_5 | 103.9 | 2019-12-02 |
| Southampton - Background (near city centre, AURN site) | 2019-12-03 01:00:00 | pm2_5 | 90.9 | 2019-12-03 |
p <- makeDotPlot(dt[!is.na(value)], xVar = "dateTimeUTC", yVar = "value", byVar = "site", yLab = yLab)
p <- p + labs(caption = "NB: There is no WHO PM2.5 hourly threshold")
plotly::ggplotly(p) # for interactionFigure 4.9: PM2_5 levels, Southampton (hourly, last 12 months)
Figure 4.9 shows hourly values for all sites for the last 12 months while Table 4.11 reports the 6 worst hours.
5 Observations
- Why are there large data holes? (see tile plots)
- Nitorgen Dioxide levels appear to be trending downwards
- For particulates the trend is more complex and the trends for PM10 and PM2.5 differ.
- Something happened on the 27th October 2019 at 20:00. There are spikes on all hourly PM plots (although this is masked in the daily plots). Could it have been a cruise ship leaving?
- Something else happened on the 2nd December 2019 at 21:00. Was this another ship?
6 Annex
6.1 Original data
skimr::skim(origDataDT)| Name | origDataDT |
| Number of rows | 262398 |
| Number of columns | 10 |
| _______________________ | |
| Column type frequency: | |
| character | 1 |
| logical | 1 |
| numeric | 7 |
| POSIXct | 1 |
| ________________________ | |
| Group variables | None |
Variable type: character
| skim_variable | n_missing | complete_rate | min | max | empty | n_unique | whitespace |
|---|---|---|---|---|---|---|---|
| site | 0 | 1 | 22 | 54 | 0 | 6 | 0 |
Variable type: logical
| skim_variable | n_missing | complete_rate | mean | count |
|---|---|---|---|---|
| co | 262398 | 0 | NaN | : |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| no | 135024 | 0.49 | 26.32 | 40.01 | -5.0 | 4.6 | 13.1 | 31.8 | 746.3 | ▇▁▁▁▁ |
| no2 | 135051 | 0.49 | 36.75 | 22.47 | -1.3 | 20.2 | 32.8 | 48.7 | 287.5 | ▇▂▁▁▁ |
| nox | 179813 | 0.31 | 76.21 | 74.88 | -1.4 | 32.3 | 56.5 | 93.8 | 1431.8 | ▇▁▁▁▁ |
| pm10 | 202848 | 0.23 | 18.56 | 13.06 | -3.5 | 10.8 | 15.7 | 23.0 | 761.6 | ▇▁▁▁▁ |
| oz | 232908 | 0.11 | 40.88 | 23.14 | -0.2 | 24.2 | 41.7 | 56.7 | 174.1 | ▆▇▂▁▁ |
| pm2_5 | 234125 | 0.11 | 12.01 | 9.48 | -4.0 | 6.5 | 9.3 | 14.3 | 289.2 | ▇▁▁▁▁ |
| so2 | 233092 | 0.11 | 2.97 | 3.27 | -1.1 | 1.0 | 1.7 | 3.5 | 50.5 | ▇▁▁▁▁ |
Variable type: POSIXct
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| dateTimeUTC | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 43733 |
6.2 Long form of original data
skimr::skim(lDT)| Name | lDT |
| Number of rows | 1836786 |
| Number of columns | 5 |
| _______________________ | |
| Column type frequency: | |
| character | 1 |
| Date | 1 |
| factor | 1 |
| numeric | 1 |
| POSIXct | 1 |
| ________________________ | |
| Group variables | None |
Variable type: character
| skim_variable | n_missing | complete_rate | min | max | empty | n_unique | whitespace |
|---|---|---|---|---|---|---|---|
| site | 0 | 1 | 22 | 54 | 0 | 6 | 0 |
Variable type: Date
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| obsDate | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 1827 |
Variable type: factor
| skim_variable | n_missing | complete_rate | ordered | n_unique | top_counts |
|---|---|---|---|---|---|
| variable | 0 | 1 | FALSE | 7 | co: 262398, no2: 262398, nox: 262398, oz: 262398 |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| value | 1480235 | 0.19 | 38.45 | 46.01 | -4 | 12.6 | 27.1 | 49.7 | 1431.8 | ▇▁▁▁▁ |
Variable type: POSIXct
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| dateTimeUTC | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 43733 |
6.3 Nitrogen Dioxide
t <- lDT[variable == "no2"]
skimr::skim(t)| Name | t |
| Number of rows | 262398 |
| Number of columns | 5 |
| _______________________ | |
| Column type frequency: | |
| character | 1 |
| Date | 1 |
| factor | 1 |
| numeric | 1 |
| POSIXct | 1 |
| ________________________ | |
| Group variables | None |
Variable type: character
| skim_variable | n_missing | complete_rate | min | max | empty | n_unique | whitespace |
|---|---|---|---|---|---|---|---|
| site | 0 | 1 | 22 | 54 | 0 | 6 | 0 |
Variable type: Date
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| obsDate | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 1827 |
Variable type: factor
| skim_variable | n_missing | complete_rate | ordered | n_unique | top_counts |
|---|---|---|---|---|---|
| variable | 0 | 1 | FALSE | 1 | no2: 262398, co: 0, nox: 0, oz: 0 |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| value | 135051 | 0.49 | 36.75 | 22.47 | -1.3 | 20.2 | 32.8 | 48.7 | 287.5 | ▇▂▁▁▁ |
Variable type: POSIXct
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| dateTimeUTC | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 43733 |
t <- lDT[variable == "no2" & value > hourlyNo2Threshold_WHO][order(-value)]
kableExtra::kable(caption = paste0("Values greater than WHO threshold (NO2 > ", hourlyNo2Threshold_WHO,
")"), t) %>% kable_styling()| site | dateTimeUTC | variable | value | obsDate |
|---|---|---|---|---|
| Southampton - Victoria Road (Woolston) | 2016-11-30 17:00:00 | no2 | 287.5 | 2016-11-30 |
| Southampton - Victoria Road (Woolston) | 2016-11-30 19:00:00 | no2 | 275.3 | 2016-11-30 |
| Southampton - Victoria Road (Woolston) | 2016-11-30 18:00:00 | no2 | 268.7 | 2016-11-30 |
| Southampton - Victoria Road (Woolston) | 2017-01-05 18:00:00 | no2 | 251.6 | 2017-01-05 |
| Southampton - Victoria Road (Woolston) | 2017-01-05 17:00:00 | no2 | 247.3 | 2017-01-05 |
| Southampton - Victoria Road (Woolston) | 2017-01-24 18:00:00 | no2 | 243.5 | 2017-01-24 |
| Southampton - Victoria Road (Woolston) | 2017-01-24 19:00:00 | no2 | 242.0 | 2017-01-24 |
| Southampton - Onslow Road (near RSH) | 2016-03-08 09:00:00 | no2 | 241.3 | 2016-03-08 |
| Southampton - Victoria Road (Woolston) | 2017-01-03 08:00:00 | no2 | 234.4 | 2017-01-03 |
| Southampton - Victoria Road (Woolston) | 2017-01-03 09:00:00 | no2 | 232.8 | 2017-01-03 |
| Southampton - Victoria Road (Woolston) | 2017-01-05 19:00:00 | no2 | 226.6 | 2017-01-05 |
| Southampton - Victoria Road (Woolston) | 2016-11-30 16:00:00 | no2 | 222.6 | 2016-11-30 |
| Southampton - Victoria Road (Woolston) | 2016-12-29 09:00:00 | no2 | 218.7 | 2016-12-29 |
| Southampton - Onslow Road (near RSH) | 2016-03-08 08:00:00 | no2 | 216.0 | 2016-03-08 |
| Southampton - Victoria Road (Woolston) | 2017-01-05 20:00:00 | no2 | 214.3 | 2017-01-05 |
| Southampton - Victoria Road (Woolston) | 2017-01-24 20:00:00 | no2 | 212.7 | 2017-01-24 |
| Southampton - Victoria Road (Woolston) | 2016-11-30 09:00:00 | no2 | 209.9 | 2016-11-30 |
| Southampton - Onslow Road (near RSH) | 2016-03-12 22:00:00 | no2 | 208.5 | 2016-03-12 |
| Southampton - Onslow Road (near RSH) | 2016-03-08 07:00:00 | no2 | 208.1 | 2016-03-08 |
| Southampton - Victoria Road (Woolston) | 2016-11-30 20:00:00 | no2 | 207.0 | 2016-11-30 |
| Southampton - Onslow Road (near RSH) | 2016-02-19 08:00:00 | no2 | 206.3 | 2016-02-19 |
| Southampton - Victoria Road (Woolston) | 2016-01-19 19:00:00 | no2 | 204.4 | 2016-01-19 |
| Southampton - Onslow Road (near RSH) | 2016-03-21 08:00:00 | no2 | 202.8 | 2016-03-21 |
6.4 PM 10
t <- lDT[variable == "pm10"]
skimr::skim(t)| Name | t |
| Number of rows | 262398 |
| Number of columns | 5 |
| _______________________ | |
| Column type frequency: | |
| character | 1 |
| Date | 1 |
| factor | 1 |
| numeric | 1 |
| POSIXct | 1 |
| ________________________ | |
| Group variables | None |
Variable type: character
| skim_variable | n_missing | complete_rate | min | max | empty | n_unique | whitespace |
|---|---|---|---|---|---|---|---|
| site | 0 | 1 | 22 | 54 | 0 | 6 | 0 |
Variable type: Date
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| obsDate | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 1827 |
Variable type: factor
| skim_variable | n_missing | complete_rate | ordered | n_unique | top_counts |
|---|---|---|---|---|---|
| variable | 0 | 1 | FALSE | 1 | pm1: 262398, co: 0, no2: 0, nox: 0 |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| value | 202848 | 0.23 | 18.56 | 13.06 | -3.5 | 10.8 | 15.7 | 23 | 761.6 | ▇▁▁▁▁ |
Variable type: POSIXct
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| dateTimeUTC | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 43733 |
kableExtra::kable(caption = paste0("PM 10 values greater than WHO threshold (NO2 > ", hourlyNo2Threshold_WHO,
")"), extremePm10Daily) %>% kable_styling()| obsDate | site | mean |
|---|---|---|
| 2016-08-09 | Southampton - A33 Roadside (near docks, AURN site) | 144.48125 |
| 2017-06-19 | Southampton - A33 Roadside (near docks, AURN site) | 143.66000 |
| 2017-05-05 | Southampton - A33 Roadside (near docks, AURN site) | 88.94348 |
| 2016-09-19 | Southampton - A33 Roadside (near docks, AURN site) | 88.45625 |
| 2016-11-30 | Southampton - A33 Roadside (near docks, AURN site) | 79.25417 |
| 2016-03-12 | Southampton - A33 Roadside (near docks, AURN site) | 72.00000 |
| 2017-01-22 | Southampton - A33 Roadside (near docks, AURN site) | 66.86250 |
| 2019-02-27 | Southampton - Background (near city centre, AURN site) | 64.67143 |
| 2019-02-27 | Southampton - A33 Roadside (near docks, AURN site) | 60.59524 |
| 2019-04-17 | Southampton - Background (near city centre, AURN site) | 58.37083 |
| 2016-12-01 | Southampton - A33 Roadside (near docks, AURN site) | 54.80833 |
| 2017-01-26 | Southampton - A33 Roadside (near docks, AURN site) | 53.90000 |
| 2019-03-29 | Southampton - Background (near city centre, AURN site) | 53.69167 |
| 2017-12-19 | Southampton - Background (near city centre, AURN site) | 53.66250 |
| 2016-12-06 | Southampton - A33 Roadside (near docks, AURN site) | 53.45000 |
| 2019-04-17 | Southampton - A33 Roadside (near docks, AURN site) | 53.08750 |
| 2019-04-07 | Southampton - Background (near city centre, AURN site) | 52.94167 |
| 2018-04-21 | Southampton - Background (near city centre, AURN site) | 52.32083 |
| 2019-03-30 | Southampton - Background (near city centre, AURN site) | 51.66250 |
| 2019-04-22 | Southampton - Background (near city centre, AURN site) | 50.86250 |
| 2019-02-23 | Southampton - Background (near city centre, AURN site) | 50.21667 |
6.5 PM 2.5
t <- lDT[variable == "pm2_5"]
skimr::skim(t)| Name | t |
| Number of rows | 262398 |
| Number of columns | 5 |
| _______________________ | |
| Column type frequency: | |
| character | 1 |
| Date | 1 |
| factor | 1 |
| numeric | 1 |
| POSIXct | 1 |
| ________________________ | |
| Group variables | None |
Variable type: character
| skim_variable | n_missing | complete_rate | min | max | empty | n_unique | whitespace |
|---|---|---|---|---|---|---|---|
| site | 0 | 1 | 22 | 54 | 0 | 6 | 0 |
Variable type: Date
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| obsDate | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 1827 |
Variable type: factor
| skim_variable | n_missing | complete_rate | ordered | n_unique | top_counts |
|---|---|---|---|---|---|
| variable | 0 | 1 | FALSE | 1 | pm2: 262398, co: 0, no2: 0, nox: 0 |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| value | 234125 | 0.11 | 12.01 | 9.48 | -4 | 6.5 | 9.3 | 14.3 | 289.2 | ▇▁▁▁▁ |
Variable type: POSIXct
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| dateTimeUTC | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 43733 |
kableExtra::kable(caption = paste0("PM 2.5 values greater than WHO threshold (NO2 > ", hourlyNo2Threshold_WHO,
")"), extremePm25Daily) %>% kable_styling()| obsDate | site | mean |
|---|---|---|
| 2016-03-12 | Southampton - Background (near city centre, AURN site) | 64.79167 |
| 2019-04-17 | Southampton - Background (near city centre, AURN site) | 49.04167 |
| 2016-03-13 | Southampton - Background (near city centre, AURN site) | 47.79167 |
| 2019-04-07 | Southampton - Background (near city centre, AURN site) | 46.90417 |
| 2019-02-27 | Southampton - Background (near city centre, AURN site) | 43.63810 |
| 2019-03-29 | Southampton - Background (near city centre, AURN site) | 43.59583 |
| 2019-04-22 | Southampton - Background (near city centre, AURN site) | 42.92500 |
| 2017-02-12 | Southampton - Background (near city centre, AURN site) | 42.24583 |
| 2018-04-21 | Southampton - Background (near city centre, AURN site) | 41.96250 |
| 2016-03-11 | Southampton - Background (near city centre, AURN site) | 41.80000 |
| 2019-04-16 | Southampton - Background (near city centre, AURN site) | 41.77500 |
| 2019-03-30 | Southampton - Background (near city centre, AURN site) | 40.80000 |
| 2017-12-19 | Southampton - Background (near city centre, AURN site) | 40.45833 |
| 2016-01-20 | Southampton - Background (near city centre, AURN site) | 39.66667 |
| 2017-09-26 | Southampton - Background (near city centre, AURN site) | 39.12083 |
| 2017-03-27 | Southampton - Background (near city centre, AURN site) | 38.15833 |
| 2019-04-08 | Southampton - Background (near city centre, AURN site) | 37.65833 |
| 2019-04-18 | Southampton - Background (near city centre, AURN site) | 37.00417 |
| 2019-02-15 | Southampton - Background (near city centre, AURN site) | 36.48571 |
| 2019-02-23 | Southampton - Background (near city centre, AURN site) | 35.80833 |
| 2018-05-08 | Southampton - Background (near city centre, AURN site) | 35.52917 |
| 2018-05-07 | Southampton - Background (near city centre, AURN site) | 35.44583 |
| 2017-02-13 | Southampton - Background (near city centre, AURN site) | 35.42083 |
| 2018-05-06 | Southampton - Background (near city centre, AURN site) | 34.07917 |
| 2016-03-17 | Southampton - Background (near city centre, AURN site) | 33.73333 |
| 2018-04-11 | Southampton - Background (near city centre, AURN site) | 33.45000 |
| 2019-01-24 | Southampton - Background (near city centre, AURN site) | 33.33750 |
| 2018-04-14 | Southampton - Background (near city centre, AURN site) | 33.20417 |
| 2018-12-27 | Southampton - Background (near city centre, AURN site) | 33.03333 |
| 2018-11-22 | Southampton - Background (near city centre, AURN site) | 32.90833 |
| 2017-02-11 | Southampton - Background (near city centre, AURN site) | 32.72917 |
| 2017-04-08 | Southampton - Background (near city centre, AURN site) | 32.65833 |
| 2019-12-02 | Southampton - Background (near city centre, AURN site) | 32.61667 |
| 2020-01-21 | Southampton - Background (near city centre, AURN site) | 32.60417 |
| 2016-03-23 | Southampton - Background (near city centre, AURN site) | 32.31667 |
| 2018-11-23 | Southampton - Background (near city centre, AURN site) | 32.30417 |
| 2019-12-03 | Southampton - Background (near city centre, AURN site) | 32.15417 |
| 2018-05-26 | Southampton - Background (near city centre, AURN site) | 32.12083 |
| 2017-11-03 | Southampton - Background (near city centre, AURN site) | 31.53333 |
| 2019-04-15 | Southampton - Background (near city centre, AURN site) | 31.52917 |
| 2017-04-09 | Southampton - Background (near city centre, AURN site) | 31.25417 |
| 2019-04-23 | Southampton - Background (near city centre, AURN site) | 31.11250 |
| 2016-01-01 | Southampton - Background (near city centre, AURN site) | 30.73750 |
| 2017-03-24 | Southampton - Background (near city centre, AURN site) | 30.70833 |
| 2019-02-24 | Southampton - Background (near city centre, AURN site) | 30.55417 |
| 2018-11-24 | Southampton - Background (near city centre, AURN site) | 30.48333 |
| 2017-09-27 | Southampton - Background (near city centre, AURN site) | 30.29583 |
| 2018-04-12 | Southampton - Background (near city centre, AURN site) | 30.21667 |
| 2017-11-02 | Southampton - Background (near city centre, AURN site) | 30.10833 |
| 2019-03-31 | Southampton - Background (near city centre, AURN site) | 29.72500 |
| 2019-10-27 | Southampton - Background (near city centre, AURN site) | 29.64167 |
| 2018-10-05 | Southampton - Background (near city centre, AURN site) | 29.54167 |
| 2019-02-25 | Southampton - Background (near city centre, AURN site) | 29.40000 |
| 2018-03-25 | Southampton - Background (near city centre, AURN site) | 29.12083 |
| 2019-04-21 | Southampton - Background (near city centre, AURN site) | 28.97083 |
| 2017-02-14 | Southampton - Background (near city centre, AURN site) | 28.96250 |
| 2019-03-28 | Southampton - Background (near city centre, AURN site) | 28.46250 |
| 2019-04-19 | Southampton - Background (near city centre, AURN site) | 28.36250 |
| 2016-01-19 | Southampton - Background (near city centre, AURN site) | 28.12917 |
| 2017-02-10 | Southampton - Background (near city centre, AURN site) | 27.43333 |
| 2017-08-28 | Southampton - Background (near city centre, AURN site) | 27.42500 |
| 2019-01-04 | Southampton - Background (near city centre, AURN site) | 27.05000 |
| 2018-11-05 | Southampton - Background (near city centre, AURN site) | 26.94167 |
| 2019-02-26 | Southampton - Background (near city centre, AURN site) | 26.87391 |
| 2018-01-12 | Southampton - Background (near city centre, AURN site) | 26.85417 |
| 2018-12-28 | Southampton - Background (near city centre, AURN site) | 26.82500 |
| 2016-05-07 | Southampton - Background (near city centre, AURN site) | 26.77500 |
| 2019-01-21 | Southampton - Background (near city centre, AURN site) | 26.65000 |
| 2019-04-20 | Southampton - Background (near city centre, AURN site) | 26.57917 |
| 2017-06-21 | Southampton - Background (near city centre, AURN site) | 26.56667 |
| 2017-03-28 | Southampton - Background (near city centre, AURN site) | 26.47500 |
| 2019-04-06 | Southampton - Background (near city centre, AURN site) | 26.32917 |
| 2020-01-20 | Southampton - Background (near city centre, AURN site) | 26.30417 |
| 2020-01-22 | Southampton - Background (near city centre, AURN site) | 26.27083 |
| 2018-05-20 | Southampton - Background (near city centre, AURN site) | 26.03333 |
| 2018-05-24 | Southampton - Background (near city centre, AURN site) | 25.93750 |
| 2017-11-01 | Southampton - Background (near city centre, AURN site) | 25.77500 |
| 2016-05-12 | Southampton - Background (near city centre, AURN site) | 25.75000 |
| 2018-04-19 | Southampton - Background (near city centre, AURN site) | 25.75000 |
| 2017-06-20 | Southampton - Background (near city centre, AURN site) | 25.74167 |
| 2018-05-29 | Southampton - Background (near city centre, AURN site) | 25.70833 |
| 2019-12-04 | Southampton - Background (near city centre, AURN site) | 25.51304 |
| 2017-05-06 | Southampton - Background (near city centre, AURN site) | 25.04167 |
7 Runtime
Report generated using knitr in RStudio with R version 3.6.3 (2020-02-29) running on x86_64-apple-darwin15.6.0 (Darwin Kernel Version 19.3.0: Thu Jan 9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64).
t <- proc.time() - startTime
elapsed <- t[[3]]Analysis completed in 74.173 seconds ( 1.24 minutes).
R packages used:
- data.table - (Dowle et al. 2015)
- ggplot2 - (Wickham 2009)
- here - (Müller 2017)
- kableExtra - (Zhu 2018)
- lubridate - (Grolemund and Wickham 2011)
- plotly - (Sievert et al. 2016)
- skimr - (Arino de la Rubia et al. 2017)
References
Arino de la Rubia, Eduardo, Hao Zhu, Shannon Ellis, Elin Waring, and Michael Quinn. 2017. Skimr: Skimr. https://github.com/ropenscilabs/skimr.
Dowle, M, A Srinivasan, T Short, S Lianoglou with contributions from R Saporta, and E Antonyan. 2015. Data.table: Extension of Data.frame. https://CRAN.R-project.org/package=data.table.
Grolemund, Garrett, and Hadley Wickham. 2011. “Dates and Times Made Easy with lubridate.” Journal of Statistical Software 40 (3): 1–25. http://www.jstatsoft.org/v40/i03/.
Müller, Kirill. 2017. Here: A Simpler Way to Find Your Files. https://CRAN.R-project.org/package=here.
Sievert, Carson, Chris Parmer, Toby Hocking, Scott Chamberlain, Karthik Ram, Marianne Corvellec, and Pedro Despouy. 2016. Plotly: Create Interactive Web Graphics via ’Plotly.js’. https://CRAN.R-project.org/package=plotly.
Wickham, Hadley. 2009. Ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York. http://ggplot2.org.
Zhu, Hao. 2018. KableExtra: Construct Complex Table with ’Kable’ and Pipe Syntax. https://CRAN.R-project.org/package=kableExtra.